Dynamic Structure from Motion using Uncalibrated Cameras and Unsegmented Scenes

Much work has been done in the last decade by the Computer Vision community in understanding the geometry of images of a rigid scene taken by a moving camera. The case of a scene containing motion has been largely ignored. Since most video footage aim at capturing events, hence motion, the need for handling dynamic scenes became apparent. Our work deals with discovering the geometrical models and the mathematical tools that we can use to analyze views of such scenes. In particular, we focus on the extraction of information about multiple independently moving objects. Unlike previous work which at best tried to ignore such moving objects, we show that valuable information can be extracted from such motions. The main mathematical tools that we have used are projective algebra and multi-linear tensors. Projective algebra has long been used to model the process of imaging of rigid scenes. From this model multiple views’ invariants can be derived to describe for example stereo vision. It was shown that these invariants can be described most generally by using multi-linear tensors. In our work, we further use multi-linear tensors to model dynamic scenes. In order to handle dynamic scenes we have often lifted the model of the scene to higher projective spaces. In these higher spaces, we were able to derive novel multi-linear invariants. We then found ways to decompose these invariants to unravel underlying information such as scene structure, motion in the scene and the motion of the camera. The research presented in this thesis appears in the following papers [46, 57, 33, 56, 58, 61]. These papers are reprinted as chapters of this thesis: Chapter 2 describes the recovery of structure and motion of a scene containing points moving along coplanar lines, appeared in ECCV2000 European Conference on Computer Vision [46]. Chapter 3 describes the recovery of structure and motion of a scene containing points in 3D moving in constant velocity, and other scenarios, appeared in International Journal on Computer Vision (IJCV) 48(1), 2002. [57]. Chapter 4 describes the derivation multi linear constraints used to recognize a moving object from a single view appeared in CVPR 2001 IEEE Conf. on Computer Vision and Pattern Recognition [33]